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DIAGNOSTIC DEVICE, SYSTEM AND METHOD FOR REDUCED DATA 

TRANSMISSION 

FIELD OF THE INVENTION 

5 The present invention relates to an in vivo device, system and method such as for 

imaging the digestive tract; more specifically, to an in vivo device, system and method where 
information transmitted or sent, is compressed. 

BACKGROUND OF THE INVENTION 

10 Devices and methods for performing in-vivo imaging of passages or cavities within 

a body, and for gathering information other than or in addition to image information (e.g-, 
temperature information, pressure information), are known in the art. Such devices may 
include, inter alia, various endoscopic imaging systems and devices for performing 
imaging in various internal body cavities. 

15 An in-vivo imaging device may include, for example, an imaging system for 

obtaining images from inside a body cavity or lumen, such as the GI tract The imaging 
system may include, for example, an illumination unit, such as a set of light emitting 
diodes (LEDs), or other suitable light sources. The device may include an imaging sensor 
and an optical system, which focuses the images onto the imaging sensor. A transmitter 

20 and antenna may be included for transmitting the images signals. A receiver/recorder, for 
example worn by the patient, may record and store image and other data. The recorded 
data may then be downloaded from the receiver/recorder to a computer or workstation for 
display and analysis. Such imaging and other devices may transmit data such as image 
data or other data during a certain period of time. It may be desirable to limit the amount 

25 of time spent transmitting image data, and also the bandwidth required for such a 
transmission. The time spent transmitting limits the amount of image or other data that 
may be transmitted Other in-vivo diagnostic units need not transmit by radio waves, for 
example, image or other data collected may be sent via wire. 

Therefore, there is a need for an in-vivo diagnostic device, such as an imaging 

30 device, which more efficiently transmits data. 
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SUMMARY OF THE INVENTION 
An embodiment of the device, system and method of the present invention enables 
the obtaining of in vivo images from within body lumens or cavities, such as images of the 
gastrointestinal (GI) tract, where the data such as image data is typically transmitted or 
otherwise sent to a receiving system. According to one embodiment of the invention, tihe data 
transmitted, including, for example, image information, is compressed. The data may be 
reconstructed using suitable methods and, for example, displayed to a user. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 A shows a schematic diagram of an in vivo imaging system according to 
one embodiment of the present invention; 

Figure IB shows a schematic diagram of an in vivo imaging system according an 
embodiment of the present invention; 

Figure 1C shows a schematic diagram of an in vivo imaging system according an 
embodiment of the present invention; 

Figure ID shows a schematic diagram of an in vivo imaging system according to an 
embodiment of the present invention; 

Figure 2 depicts a series of steps of a method according to an embodiment of the 
present invention; 

Figure 3 depicts a schematic diagram of a first exemplary dilution pattern for 
selecting pixels according to an embodiment of the present invention; 

Figure 4 depicts a schematic diagram of a second exemplary dilution pattern for 
selecting pixels according to an embodiment of the present invention; 

Figure 5 depicts a schematic diagram of pixels of one color selected according to an 
exemplary dilution pattern in accordance with the present invention; 

Figure 6 depicts a schematic diagram of partial reconstruction of an image based on 
pixels selected according to an exemplary dilution pattern in accordance with the present 
invention; and 

Figure 7 depicts a series of steps of a method for reconstructing a diluted image 
according to an embodiment of the present invention. 
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DET AILE D DESCRIPTION OF EMBODIMENTS OF THE INVENTION 
In the following description, various aspects of the present invention will be 
described. For purposes of explanation, specific configurations and details are set forth in 
order to provide a thorough mderstanding of the present invention. However, it will also be 

5 apparent to one skilled in the art that the present invention may be practiced without the 
specific details presented herein. Furthermore, well-known features may be omitted or 
simplified in order not to obscure the present invention. 

Embodiments of the system and method of the present invention may be 
preferably used in conjunction with an imaging system or device such as described in U.S. 

io patent 5,604,531 to Iddan et aL and/or in application number WO 01/65995 entitled "A 
Device And System For In Vivo Imaging", published on 13 September, 2001, both of 
which are hereby incorporated by reference. However, the device, system and method 
according to the present invention may be used with any device providing imaging or other 
data from a body lumen or cavity. In alternate embodiments, the system and method of the 

15 present invention may be used with devices capturing information other than image 
information within the human body; for example, temperature, pressure or pH information, 
information on the location of the transmitting device, or other information. 

Reference is made to Figs. 1A-1D, which show schematic diagrams of in vivo 
imaging system according to embodiments of the present invention. In an exemplary 

20 embodiment shown in Fig. 1 A, a device 40 may be an ingestible capsule capturing images, 
but may be another sort of device and may collect information other than image information. 
Typically, device 40 may include at least one sensor such as an imager 46, for captuiing 
images, a processing chip or circuit 47 that processes the signals generated by the imager 46, 
and one or more illumination sources 42, for example one or more 'Svhite LEDs" or any other 

25 suitable light source, for illuminating the body lumen- An optical system 50, including, for 
example, one or more optical elements (not shown), such as one or more lenses or composite 
lens assemblies (not shown), one or more suitable optical filters (not shown), or any other 
suitable optical elements (not shown), may aid in focusing reflected light onto the imager 46 
and performing other light processing. Processing chip 47 need not be a separate component; 

30 for example, processing or a processing chip may be integral to the imager 46. A non-image 
sensor 49, for example, such as a temperature sensor, a pH sensor, or a pressure sensor may 
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be included. In an alternate embodiment of the present invention sensor 46 may be a 
non-image sensor such as a temperature sensor, a pH sensor, or a pressure sensor. 

Device 40 typically includes a transmitter 41, for transmitting image and possibly 
other information (e.g., control information, non-image data, etc.) to a receiving device, and a 

5 compression module 600, for compressing data. The transmitter' 41 may typically be an ultra 
low power radio frequency (RF) transmitter with high bandwidth input, possibly provided in 
chip scale packaging. The transmitter may transmit via an antenna 48. Hie transmitter 41 
may act as a controller may also include circuitry and functionality for controlling the device 
40. Typically, the device may include a power source 45, such as one or more batteries. For 

10 example, the power source 45 may include silver oxide batteries, lithium batteries, or other 
electrochemical cells having a high energy density, or the like. Other suitable power sources 
may be used. 

Ofcer components and sets of components may be used* For example, the power 
source may be an external power source transmitting power to the capsule, for example as 

15 described in a patent application with International Publication Number WO 02/080753 A2, 
and a controller separate from fee transmitter 41 may be used. 

In one embodiment, the imager 4-6 may be a complementary metal oxide 
semiconductor (CMOS) imaging camera The CMOS imager may typically be an ultra low 
power imager and may be provided in chip scale packaging (CSP). One suitable CMOS 

20 camera may be, for example, a "camera on a chip" CMOS imager specified by Given 
Imaging Ltd. of Israel and designed by Photobit Corp. of California, USA, with integrated 
active pixel and post processing circuitry. Other types of CMOS imagers may be used. In 
another embodiment, another imager may be used, such as a CCD imager, or another imager. 

Typically, the device 40 is swallowed by a patient and traverses a patient's GI tract, 

25 however, other body lumens or cavities, such as blood vessels, the female reproductive tract, 
etc., may be imaged or examined The device 40 transmits image and possibly other data to 
components located outside the patient's body, which may receive and process the data. 
Preferably, located outside the patient's body in one or more locations, may be a receiver 12, 
preferably including or attached to an antenna or antenna array 15, for receiving image and 

30 possibly other data from device 40, and a controller or processor 113, a receiver storage unit 
16, for storing image and other data, a data processor 14, a data processor storage unit 19, a 
data decompression module 610 for decompressing data, and an image monitor 18, for 
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displaying, inter alia, the images or reconstructed versions of the images ttansmitted by the 
device 40 and recorded by the receiver 12. Typically, the receiver 12 and receiver storage 
unit 16 may be small and portable, and may be worn on the patient's body during recording of 
the images. Preferably, data processor 14, data processor storage nnit 19 and monitor IS may 

5 be part of a personal computer or workstation, which includes, for example, standard 
components such as a processor 13, a memory (e.g, storage 19, or other memory), a disk 
drive, and input-output devices, although alternate configurations are possible. In alternate 
embodiments, the data reception and storage components may be of another suitable 
configuration. Further, image and other data may be received in other manners, by other sets 

10 of components. Typically, in operation, image data may be transferred to the data processor 
14, which, in conjunction with processor 13 and software, may store, possibly process, and 
display the image data on monitor 18. Other systems and methods of storing and/or 
displaying collected image data may be used. Any of data processor 14, processor 13, 
receiver 12, controller 1 1 3, or another component or set of components, may act as or include 

15 a suitable controller or processor for, inter alia, controlling the receipt and transfer of data, 
reconstructing data, decompressing data, etc. 

Typically, the device 40 transmits image information in discrete portions. Each 
portion typically corresponds to an image or frame. Other transmission methods are possible. 
For example, the device 40 may capture an image once every half second, and, after capturing 

20 such an image, may transmit the image to the receiving antenna. Other capture rates may be 
possible- Typically, the image data recorded and transmitted may be a digital color image 
data, although in alternate embodiments other image formats (e.g., black and white image 
data) may be used. In one embodiment, each frame of image data includes 256 rows of 256 
pixels each, each pixel including data for color and brightness, according to known methods. 

25 For example, in each pixel, color may be represented by a mosaic of four sub-pixels, each 
sub-pixel corresponding to primaries such as red, green, or blue (where one primary is 
represented twice). In other embodiments, each pixel may capture only one color. The 
brightness of the overall pixel may be recorded by, for example, a one byte (z.e., 0-255) 
brightness value. Other data formats may be used. 

30 In some embodiments of the device, system and method of the present invention, 

diagnostic data need not be transmitted, but may be sent via another method, such as via wire. 
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For example, in an endoscope device, an imaging device at one end may send the data via 
wire to a receiving device. 

It may be desirable to limit the amount of time spent transmitting image data, and/or 
the bandwidth required for such a transmission. Embodiments of the system and method of 
the present invention may compress image and possibly other data before tra nsm i s sion. Since 
compressed data may take less time to transmit, more data may be transmitted, and more 
frames of image data may be transmitted per time unit, without increasing, foe example, the 
bandwidth of the transmitter Alternatively, the same amount of data may be transmitted 
using less bandwidth. Another aspect of the data transmission relates to the transmission 
systems with limited energy source. In this case smaller amount of bits needed to be 
transmitted may enable more energy per bit in the transmission. Data other than or in addition 
to image data may be transmitted and compressed. For example, control information may be 
compressed Furthermore, in devices transmitting telemetric information other than image 
information, such as pressure or pH information, such information may be compressed- In 
further embodiments, image data need not be transmitted in discrete portions corresponding to 
images. 

Thus, for example, if the bandwidth of a transmission mechanism permits an 
uncompressed frame relay rate of, for example, two frames per second at a specified bit rate, 
the same tr ans mission mechanism may be able to support the transmission of a greater 
number of frames per second using the same bit rate if the transmitted data is compressed or 
diluted prior to compression, and then reconstructed after transmission. Thus, according to 
one embodiment, for areas of the gastro-intestinal tract where a greater number of frames per 
second may be desired (for example, the esophagus, which may be traversed quickly by a 
capsule), the imaging device may operate in a "fast mode" which transmits compressed or 
diluted data and may therefore be capable of transmitting a greater number of frames per 
second than uncompressed data. In one embodiment, data may be added by for example 
interpolation and/or other methods to produce completed images or images that may be 
presented as completed images. 

In an exemplary embodiment of the present invention, device 40 includes a data 
compression module 600 for compressing data transmittal from the device 40 and for 
providing the data to the transmitter 41, possibly via intermediate circuitry. Data compression 
module 600 may be implemented as part of a microprocessor or ASIC or other 
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microcomputing device, as part of the imager 46 or processing chip 47, or in another suitable 
manner. In alternate embodiments the functions of the data compression module 600 may be 
taken up by other structures and may be disposed in different, parts of the device 40. For 
example, the transmitter 41 may include data compression. capability, or data compression 

5 module 600 may be a stand-alone unit, or may be implemented in software. In one 
embodiment, transmitter 41 may include, for example, a modulator 70 for receiving the video 
signal from the imager 46, a radio frequency (RF) amplifier 72, and an impedance matcher 
74. The modulator may convert the input image signal having a cutoff frequency o£ for 
example, f c of less than 5 MHz to an RF signal having a carrier frequency f r , typically in the 

10 range of 1 GHz (other ranges may be used). While in one embodiment tie signal is an analog 
video signal, the modulating signal may be another signal, for ©cample digital rather than 
analog. The carrier frequency may be in other bands, e.g. a 400MHz band. The modulated 
RF signal may have a bandwidth of The impedance matcher may match the impedance of 
the circuit to that of the antenna Other suitable transmitters or arrangements of transmitter 

15 components may be used, utilizing different signal formats and frequenc}' ranges. For 
example, alternate embodiments may not include a matched antenna or may include a 
transmitter without a matching circuit In one embodiment of such an imaging device 40, 
transmission may occur at a frequency of 434MHz, using Phase Shift Keying (PSK). In 
alternate embodiments, other transmission frequencies and methods (such as AM or FM) may 

20 be used. 

The receiver 12 may preferably detect a signal having the carrier frequency f r and the 
bandwidth £ described herein. The receiver 12 may be similar to those found in televisions or 
for example it may be one similar to those described on pages 244-245 of the book 
Biomedical Telemetry by R, Stewart McKay and published by John Wiley and Sons, 1970. 
25 The receiver may be digital or analog. In alternate embodiments, other receivers, responding 
to other types of signals, may be used 

The receiver 12 preferably includes a data decompression module 610 for 
decompressing data received from the device 40. In exemplary embodiment data 
decompression module 610 may be a microprocessor or other micro-computing device and 
30 may be part of the receiver 12. In alternate embodiments the functions of the data 
decompression (decoding) module 610 may be taken up by other structures and may be 
disposed in different parts or more than one part of the system; for example, data 
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decompression module 610 may be implemented in software and/or be part of data processor 
14. The receiver 12 may receive compressed data without decompressing the data and store 
the compressed data in the receiver storage unit 16. The data may be later decompressed by, - 
for example data processor 14. 

5 Preferably, the transmitter 41 may provide overall control of the device 40; in 

alternate embodiments control may be provided by other modules. Preferably, the data 
compression module 600 may interface with the transmitter 41 to receive and compress 
image data; other units may provide other data to data compression module 600. In 
addition, the data compression module 600 may provide the transmitter 41 with 

10 information such as, for example, start or stop time for the transfer of image data from the 
data compression module 600 to the transmitter 41, the length or size of each block of such 
image data, and the rate of frame data transfer. The interface between the data 
compression module 600 and the transmitter 41 may be handled, for example, by the data 
compression module 600. Typically, the data compression module 600 may compress 

is image infbimation in discrete portions. Each portion may typically correspond to an image 
or frame. Other compression methods or sequences are possible, and other units of 
compression and transmission are possible. In one embodiment, to compress the image 
data, subsequent images may be compared, and to only differences between these images 
may be transmitted rather than each image. Assuming that in most cases the subsequent 

20 images may be similar, the difference between the images may contain much less 
information than the image itself. 

3ii alternate embodiments, the data exchanged between the data compression module 
600 and the transmitter 41 may be different, and in different forms. For example, size 
information need not be transferred. Furthermore, in embodiments having alternate 

25 arrangements of components, the interface and protocol between the various components may 
also differ. For example, in an embodiment where a data compression capability is included 
in the transmitter 41 and the imager 46 transfers un-compressed data to the transmitter 41, no 
start/stop or size information may be transferred. In another embodiment of the invention, a 
data compression module 601 may be implemented as part of an imager 46', such as in for 

30 example, device 40 shown in Fig. IB. In one embodiment of the invention, imager 46" or 
another component within device 40 may produce a selection of input data to form a diluted 
image. In such a case, a transmitter 4T may be a transmitter without compression capability; 
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however, using selections of data and compression may be performed together. In another 
embodiment of the invention, a compression module 602 may be part of a processing chip 
47'such as in for example, device 40 shown in Fig. 1C. Processing chip 47 and AT may be 
for example, a microprocessor, ASIC, or another suitable micro-computing device. In yet 

5 another embodiment of the invention, a compression module 603 may be included as a 
stand-alone unit such as in for example, device 40 shown in Fig. ID. 

The data compression module for example, module 600 (or 601, 602 or 603), and 
data decompression module 610, may use various data compression formats and systems. 
Compression formats used may include compression formats where some data may be lost 

io during compression and compression formats where data may not be lost during 
compression. Typically, Hie data compression module 600 and decompression module 610 
may include circuitry and/or software to perform such data compression. For example, if 
the data compression module 600 or decompression module 610 (or other compression or 
decompression systems) are implemented as a computer on a chip or ASIC, data 

15 compression module 600 or decompression module 61 0 may include a processor operating 
on firmware which includes instructions for a data compression algorithm. If data 
decompression module 610 is implemented as part of data processor 14 and/or processor 
13, the decompression may be implemented as part of a software program. It will be 
evident to those of skill in the art that compression module need not be a physically 

20 separate component, but rather, that its functionality may be performed by another 
component, such as imager 46' , 

The amount of imager data to be sent may be, for example, over 1,35 Megabits 
per second. Compression may reduce this amount. After compression, and before 
transmission, other operations such as randomization may occur (performed, for example, 

25 by the transmitter 41). For example, he occurrence of the digital signals ("0" and "1") may 
be randomized so that transmission may not be impeded by a recurring signal of one type. 

Fig. 2 depicts a series of steps of a method according to an embodiment of the 
present invention. Referring to Fig. 2, in block 200, an in-vivo device, such as a 
swallowable capsule, captures image data. Typically, an imager within the device may 

30 capture image data of a gastrointestinal tract, but other image data may be captured, for 
example, image data from other body lumens or cavities. Data other than or in addition to 
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image data may be captured. The image data from the imager may be passed to other units 
for processing. 

In block 210, the image data is compressed. Such compression or other processing 
may be performed by a processing unit such as, for example, transmitter 41. The image 
5 data is typically received from the imager or via another unit Such compression may be 
typically in response to a process receiving input data corresponding to an image. In the 
embodiment shown in Fig. 2, compression may be accomplished by diluting the captured 
data, or by selecting only a pattern of pixels for transmission. Such creation of a selection 
of data, where the selection is typically less data than the original data, may be, for 

10 example, performed according to a dilution pattern. Alternately, only the differences 
between for example, pixels or regions of subsequent images may be transferred. The 
image data may be first loaded or transferred from the imager to a compression module, or, 
alternately or additionally, may be compressed at the imager. If applicable, data other than 
or in addition to image data may be compressed; in block 215, for example, sensor data 

15 other than image sensor data, control data, etc. Lei one embodiment of the invention, for 
example, data may be selected for transmission by the imager and then further compressed, 
for example, by JPEG or another algorithm before transmission. The selected data is 
typically less data than the original data. 

In block 220, the data may be transmitted to a receiver. Typically, the data, such as 

20 for example image data, may be transmitted, for example, using radio waves (RJF channel) 
to a receiver external to the body, but other methods may be used. In alternate 
embodiments, the image or other data may be sent by other methods, such as by wire. 

In block 230, the data may be decompressed, enabling reconstruction of the image. 
Reconstruction may include further elements such as pre-processing, post-processing, etc. 

25 Reconstructed image data may be, for example, displayed or stored for later use. 
Alternately, the compressed image data may be stored for later image reconstruction. 

Other suitable steps or series of steps may be used than those described in the 
above blocks. 

The data compression methods described herein may be lossless or lossy. 
30 Losseless data compression may enable precise (with no distortion) decoding of the 
compressed data. The compression ratio of loseless methods may however be limited. 
Lossy compression methods may not enable precise decoding of the compressed 
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information. However the compression ration of lossy methods may be much higher than 
of the lossless method La many cases the data distortion of the lossy methods may be 
non-significant, and the compress ratio may be high. Without limitation of generality, the 
description of data compression schemes herein may be applicable both to lossless and to 
5 lossy methods. 

In an embodiment of the present invention, compression may be accomplished by 
transmitting only portions of the captured image selected according to, for example, a dilution 
pattern. While this may result in some loss of quality in the resulting image, proper selection 
of pixels according to a suitable dilution pattern, together with proper reconstruction of the 

10 diluted image, may preserve the quality of the transmitted image and rehabilitate the image to 
lossless or near-lossless condition. The dilution pattern may for example be predetermined, 
may be selected or created by a component of the device based on operating conditions 
related, for example, to its position in the gastro-intestinal tract or other surrounding 
conditions such as pH, temperature, ambient lighting or color conditions, or may be created 

15 using other methods. Data may be selected for transmission by means other than a dilution 
pattern. 

Embodiments of the invention are presented herein with different dilution patterns, 
although those of skill in the art will recognize that other dilution patterns may be used for 
compressed transmission in accordance with the present invention. In the below exemplary 

20 dilution patterns presented, there is assumed an imager with 256 rows and 256 columns of 
pixels, each pixel representing one of the colors red, blue or green. The embodiments also 
show an imager having twice as many green pixels as red or blue pixels. It will be recognized 
by those of skill in the art that the invention may be practiced with imagers of different 
configurations, sizes, and color patterns. For example, a black and white imager may be used 

25 Fig. 3 depicts one exemplary dilution pattern in accordance with embodiments of the 

invention that may be used in an imager having, for example, pixels representing red (R) 302, 
green (G) 304, and blue (B) 306 in the arrangement shown. In this first exemplary dilution 
pattern shown, the imaging device may transmit every, for example, fourth pixel in each row, 
where, typically, all pixels chosen in any row represent the same color. In the case of three 

30 colors, e.g., red, blue and green, one color selected for transmission may repeat every second 
row, while the selection of the other two colors alternates every fourth row. Thus, for 
example, in the example shown, in two of each four consecutive rows, the color red 308 may 
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be selected, while in the remaining two rows, blue 310 and green 312 may alternate every 
fourth row. Thus, if the array of pixels in the imager includes 256 rows and 256 columns, the 
device may transmit only 64 pixels, or every fourth pixel, for each row. 

Fig. 4 depicts another exemplary dilution pattern that may be used in accordance 

5 with embodiments of the invention. In this second exemplary dilution pattern, a pattern may 
be repeated every four rows, in which no pixels may be transmitted from a first row, every 
fourth pixel may be transmitted from each of the next two rows, and every second pixel may 
be transmitted from the fourth row. In this embodiment, it will be recognized that twice as 
many green pixels 402 may be transmitted than red pixels 404 or blue pixels 406, 

io According to some embodiments of the present invention, the device may transmit 

for some or all pixels the difference between 1he actual value and a predicted value based on 
the already determined values of other pixels, for example, neighboring pixels. The receiver 
may determine the predicted value based on the values of the other pixels, and modify the 
predicted value by the difference transmitted, thereby reconstructing the original actual value 

is for the pixel. It will be reco gnize d that this embodiment of the invention may be 
implemented as lossless or lossy. For example, in one embodiment, for each pixel there may 
be transmitted either a value or an exact difference based on a predicted value for the pixeL 
In such an embodiment, no image quality in the reconstructed image will be lost In another 
example, there may be set a threshold, wherein a difference between the predicted value and 

20 the actual value that may be less than the threshold may not be transmitted. In this latter 
example, some image quality in the reconstructed image may be compromised. 

It will be recognized by those of skill in the art that while only several dilution 
patterns have been discussed at length, any suitable dilution pattern that selects some pixels or 
areas for tr ansmis sion while omitting others may be used in accordance with embodiments of 

25 this invention* In one embodiment of the invention, producing selected data includes for 
example modifying at least one input datum by reference to at least one other input datum to 
produce selected data. 

Optionally, devices in accordance with embodiments of the invention may operate in 
a "simple" fast mode, an "averaging" fast mode or other suitable mode corresponding to any 

30 suitable dilution pattern, including, for example, the modes discussed herein. In "averaging 5 ' 
mode, the value transmitted for a selected pixel of a certain color may be summed or averaged 
or otherwise affected by the value of a nearby pixel, typically a pixel of the same color. For 
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example, in Fig. 3, a pixel 308 selected by the particular dilution pattern for transmission may 
be averaged by the imager with a neighboring pixel of the same color 302 prior to 
transmission. Averaging may be performed by the compression module, for example, the 
imager, or by another component This "averaging mode" may be activated or deactivated, 
for example, by a control bit intrinsic or extrinsic to the compression module, for example, the 
imager. It will be appreciated by those of skill in the art that any modification of an input 
pixel by reference to one or more neighboring pixels, for example a weighted average, may be 
used in accordance with the present invention. 

Also optionally, in an embodiment of the present invention, there may be 
provided an error correction mechanism for detecting and correcting errors. In some 
embodiments, control or "overhead" information may be transmitted in addition to the 
pixels of each image, for example, information contained in a prefix header and/or suffix 
wori This control information may or may not be compressed Various techniques of 
error correction are known, any of which may be used in connection with any embodiment 
of the present invention. In some embodiments of the invention, the imager may perform 
error correction encoding. 

At the receiver end, various methods may be used in accordance with 
embodiments of the present invention to reconstruct an image from the diluted data 
transmitted. The method of reconstruction may vary, for example, depending on the 
dilution pattern chosen. 

In an embodiment of the invention employing the first exemplary dilution pattern, 
a full matrix of color values for each pixel may be reconstructed from the selected pixels 
using various methods. In some embodiments of tihe invention, interpolation or weighted 
interpolation between selected data may be used for reconstructing the diluted image. 
Other suitable methods for reconstructing the data may be used in addition to interpolation 
or instead of interpolation. In one embodiment of the invention, edge detection may be 
used to determine weights for weighted interpolation when reconstructing an image from a 
diluted image. For example, as shown in Fig. 5, the sampled red pixels 502 in the first 
dilution pattern may be in a rhomboid pattern. In one embodiment of the present 
invention, some or all of the values of four pixels 602 forming a rhomboid may be used to 
calculate a value for the pixel at the center of the rhomboid, as depicted by the pixels 604 
in Fig. 6. While in one embodiment of the present invention, the four values surrounding a 
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rhomboid center may be averaged, other embodiments are of course possible. For 
example, edge detection ot other suitable methods may be used to locate pixels that may be 
on the edge of an object. For pixels that may be determined to be on the edge, the center 
value may be determined by for example a weighted average of, for example, the four 
5 surrounding pixels. Whether the pixel may be on an edge may, for example, be determined 
based on the gradient at that pixel Alternately or additionally, the center value may be 
determined by a weighted average of a subset of the surrounding pixels. In one 
embodiment, the center value may be determined based on the median of some or all of the 
surrounding pixels. In one embodiment, the interpolation may be on a grid not necessarily 

10 in the shape of a square. For example, the orthogonal values between the pixels may be 
obtained by interpolation of the four surrounding pixels. Any suitable method may be used 
for this interpolation, for example, linear, quadratic, bicubic, polynomial, weighted 
average, or other interpolation. The remaining pixels, which may be located on diagonals 
between originally selected pixels, may be interpolated using known methods. 

15 Interpolating typically helps to produces additional image data, eventually resulting in a 
reconstructed image. 

With respect to the remaining two colors in the first dilution pattern, the missing 
pixels may be interpolated from the square pattern shown in Fig. 6 by any suitable method 
of interpolation, for example, linear, quadratic, bicubic, poljmomial or other interpolation. 

20 The remaining pixels, which may be located on diagonals between originally selected 
pixels, may be for example, interpolated using known methods. While only some methods 
of interpolation of missing pixels have been enumerated, it will be understood by those of 
skill in the art that any suitable method of interpolation may be used consistent with any 
embodiment of the present invention. 

25 With respect to reconstructing image data in accordance with the second 

exemplary dilution pattern, a similar reconstruction process may be used as the one 
described. Thus, for example, the rhomboid pattern created by the sampled green pixels 
may be similar to the rhomboid pattern of the red pixels in the first exemplary dilution 
pattern, and the pixels of the remaining two colors are in a square pattern. It will be 

30 understood by those of skill in the art that the present invention is not limited in the respect 
of the exemplary dilution patterns (and reconstruction schemes) described above, and that 
many others may be used in accordance with the present invention. 
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In some embodiments of the present invention, there may be further enhancement 
or refinement of the reconstructed image. In some embodiments of the present invention, 
the resulting image may be smoothed by modifying the color values of the originally 
selected pixels, for example, by replacing the original value by a weighted average of the 

5 original value taken together with some or all values of the surrounding selected pixels, for 
example, a median value of the surrounding pixels. 

The various compression and/or dilution methods discussed herein need not be 
used with a device having more than one mode, or a 'fast mode", but may be utilized for 
various other purposes. Further, compression and dilution of pixels need not be used 

10 together. 

Typically, a compression and/or dilution process is carried out by control circuitry 
in the device 40, such as transmitter 41. Similarly, reconstruction and/or decompression 
may be carried out by for example, data processor 14, decompression module 610, 
processor 13, etc., or by structures in the receiver 12. Of course, in other embodiments, 

15 such processes may be carried out by other components, and of course the methods 
discussed herein may be carried out in. devices having structures other than that of devices 
40, receiver 12, and data processor 14. For example, control processes such as producing 
the selection of input data may be carried out by an imaging component, or another 
component For example, a portion of a controller may be considered to be within the 

20 imaging unit. 

Furthermore, in some embodiments of the present invention, enhancement may be 
made by for example modifying the intensity values of the image to restore diem to near 
the original values. For example, the intensity of a reconstructed pixel may be calculated 
using only or predominantly the values of nearby originally selected pixels. In another 
25 embodiment, intensity for each pixel may be obtained by using the values of pixels of only 
one color, for example, green. It will be recognized that other methods of obtaining 
intensity values for reconstructed pixels may be used consistent with embodiments of this 
invention. 

In some embodiments of the present invention, there may be lessening of color 
30 artifacts due to the process of dilution and reconstruction of the image, for example, by 
suppressing colors of pixels located on edges found in the image. In some embodiments of 
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the invention, color suppression may also be used to correct color-saturated pixels during, 
for example, reconstruction. 

In some embodiments of the present invention, a pre-processing block may be 
added for the original samples. For example, in one embodiment, a gradient evaluation for 
enhancing edges may be added. Additionally, in one embodiment of the invention, a 
post-processing block may be added for enhancing reconstruction, for example, for 
correcting interpolation artifacts, for example, periodic artifacts. In one embodiment; these 
artifacts may be corrected in the fiequency domain by convolution or median filter. 

Fig. 7 depicts a series of steps of a method for reconstructing a diluted image or 
other suitable data according to an embodiment of the present invention, Referring to Fig. 
7, in block 700, selected data, for example a diluted image data, may be received. For 
example, the data may be received at a receiver 12, a data processor 14, or other suitable 
structure. In block 710 pre-processing may be performed on data or selected data 
Pre-processing may include for example, clearing errors with error correction code, 
reducing noise, performing a gradient evaluation for detecting and enhancing edges, 
calculating intensity, etc. Other suitable pre-processing may be used. In step 730, 
interpolation may be performed to, for example, fill in gaps between data. Interpolation 
may include, for example, linear, quadratic, bicubic, polynomial, weighted average, or 
other suitable interpolation. Edge information may be used to weight differently each of 
the samples. For example, pixels along directions with high gradient (possibly close to an 
edge) ma}', for example, not be included in interpolation. Other statable weighting factors 
may be included. In one embodiment of the invention, intensity may be calculated and 
used during interpolation for maintaining the ratio between color and intensity during 
interpolation. Other suitable interpolation methods may be implemented. After 
interpolation, post-processing may be performed on interpolated data in block 760 to, for 
example enhance reconstructed image. Post-processing for enhancing a reconstructed 
image may include, for example image sharpening, color suppression, intensity adjustment, 
convolution or a median filter. Other suitable post-processing techniques may be 
implemented. 

Embodiments of the present invention may include apparatuses for performing the 
operations herein. Such apparatuses may be specially constructed for the desired purposes 
(e.g., a "computer on a chip" or an ASIC), or may include general purpose computers 
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selectively activated or reconfigured by a computer program stored in the computers. Such 
computer programs may be stored in a computer readable storage medium, such as, but is 
not limited to, any type of disk including floppy disks, optical disks, CD-ROMs, 
magnetic-optical disks, read-only memories (ROMs), random access memories (RAMs), 
electrically programmable read-only memories (EPROMs), electrically erasable and 
programmable read only memories (EEPROMs), magnetic or optical cards, or any other 
type of media suitable for storing electronic instructions. 

The processes presented herein are not inherently related to any particular computer 
or other apparatus. Various general purpose systems may be used with programs in 
accordance with the teachings herein, or it may prove convenient to construct a more 
specialized apparatus to perform the desired method. The desired structure for a variety of 
these systems appears from the description herein. In addition, embodiments of the present 
invention are not described with reference to any particular programming language. It will 
be appreciated that a variety of programming languages may be used to implement the 
teachings of the invention as described herein. 

Unless specifically stated otherwise, as apparent from the discussions herein, it is 
appreciated that throughout the specification discussions ut ilizing terms such as 
"processing", "computing", "calculating", "dete rmining ", or the like, typically refer to the 
action and/or processes of a computer or computing system, or similar electronic 
computing device (e.g., a "computer on a chip" or ASIC), that manipulate and/or transform 
data represented as physical, such as electronic, quantities within the computing system's 
registers and/or memories into other data similarly represented as physical quantities 
within the computing system's memories, registers or other such information storage, 
transmission or display devices. 

While the invention has been described with respect to a limited number of 
embodiments, it will be appreciated that many variations, modifications and other 
applications of the invention may be made. Embodiments of the present invention may 
include other apparatuses for performing the operations herein. Such apparatuses may 
integrate the elements discussed, or may comprise alternative components to carry out the 
same purpose. It will be appreciated by persons skilled in the art that the appended claims 
are intended to cover all such modifications and changes as fall within the true spirit of the 
invention. 
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